On-line Head Pose Estimation with Binocular Hand-eye Robot Based on Evolutionary Model-based Matching On-line Head Pose Estimation with Binocular Hand-eye Robot Based on Evolutionary Model-based Matching

This paper presents a method to estimate the 3D pose of a human's head using two images input from stereo cameras. The proposed method utilizes an evolutionary search technique of 1-Step genetic algorithm (1-Step GA) being improved to adapt for real-time recognition in dynamic images and a fitness evaluation based on a stereo model matching. Here, the head position and orientation are detected simultaneously using the brightness distribution and color information of the input images, evaluating the facial features, eyebrows and eyes. Moreover, to improve the dynamics of recognition, feedforward model-based matching is proposed for hand-eye visual servoing. The effectiveness of the method is shown by the experiments to compensate the motion of the hand-eye camera against the relative motion of the object in camera frame, having resulted in the robust recognition against the hand-eye motion. INTRODUCTION This work is motivated by our desire to establish a visual system for a patient robot that is used to evaluate an ability of the medical treatments of nurse students, as shown in Fig.1. It is important for nurse to pay attention to the condition of the patient during, e.g. injection, to sense tiny sign of patient’s state so as to avoid medical accidents. What is the most important for nurses is to check the patient’s face periodically and carefully to infer their inside conditions. To evaluate this nurse abilities, the patient robot have to contrarily track the nurse’s head pose, then the patient robot can judge whether the students can give their patient a good treatment. The behaviors of patient robot to position its head pose relative to the nurse’s to observe the nurse’s head pose and gazing direction of eyes is one of visual servoing to 3D pose. There is a variety of approaches for face representation of poses, and they can be classified into three general categories: feature-based, appearance-based, and model based. Feature-based approaches use local features like points, line segments, edges, or regions. The main idea of this method is to select a set of feature points, which are matched against the incoming video to update pose estimation. In a feature-based approach, estimation based on the relationship between human facial features [1], [2] relies heavily on the accuracy of the facial feature detection schemes. Detection of facial features is not accurate and often fails because it is affected by other parameters depending on identity, distance from the camera, facial expression, noise, illumination changes, and occlusion [3], [4]. Appearance-based approaches attempt to capture and define the face as a whole. The image is compared with various templates to determine which one most closely matches the image, resulting in wasting time to recognize. This approach has received lot of attention recently. In one technique [5], templates representing facial feature are used in determining head position and orientation. The image is compared with various templates to determine which template most closely matches the image, resulting in wasting time to recognize. On the other hand, Model-based approaches is to use a model to search a target object in the image, and the model is composed based on how the target object can be seen in the input image. For recognizing a face and detecting its pose, 3-D models of face are mapped on input images to estimate the head pose [6], [7]. The recognition method developed in our paper is in this category. An